%% name       : PNWSUG2007 Paper.tex
%% description: 
%% purpose    : 
%% author     : Richard Koopmann
%% date       : 13JUL2007
%% note       :

\documentclass{sugconf}
\usepackage{textcomp}
\usepackage{listings}
  \lstloadlanguages{SAS}
  \lstset{frame=tb, basicstyle=\scriptsize \ttfamily, upquote=true}
\usepackage{graphicx}
\usepackage{floatflt}

%\pdfoutput=0%out=jobname.dvi
%\pdfoutput=1%out=jobname.pdf

\newcommand{\ODSDSO}{ODS DATA Step Object}
\newcommand{\DSCI}{DATA Step Component Interface}
\newcommand{\datanull}{\texttt{DATA \textunderscore NULL\textunderscore} }

\sugconfsubject{\ODSDSO}
%\sugconfpapernumber{PNWSUG 2007-xxx}
\sugconfpapernumber{\relax}%note: no paper number: warning in log
\sugconfkeywords{\ODSDSO, \DSCI\ (DSCI), macro, DATA NULL}
\title{Experimenting with the \ODSDSO\ (Part I)}
\author{Richard Koopmann Jr. -- Capella University -- Minneapolis, MN}

\makeatletter%necessary to copy title and author into pdf description
\usepackage[bookmarks   =false
           ,pdfauthor   ={\@author}
           ,pdfcreator  ={pdfLaTeX sugconf.cls}
           ,pdfkeywords ={\SUGconfKeywords}
           ,pdfstartview=Fit
           ,pdfsubject  ={\SUGconfSubject}
           ,pdftitle    ={\@title}
           ,colorlinks  ={true}
           ,linkcolor   =black
           ,citecolor   =black
           ,filecolor   =black
           ,urlcolor    =black
           ]{hyperref}\makeatother
\begin{document}
\pdfcompresslevel=9

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{abstract}%\section{Abstract}%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Recently, I was tasked with developing a SAS program to phase out tedious, error-prone process of transcribing a few key numbers from hundreds of item summary tables to a dozen one-page summary tables. To reduce end-user confusion, the revised reports needed to resemble the current layout and format as much as possible. Since the key statistics change across subgroups and span several item summary tables, a single tabulate or report procedure wouldn't be sufficient. It appeared that traditional ODS reporting was out.

Other reporting methods were explored, specifically \datanull reporting which allows the dynamic use of variables---via macro variables---and provides the layout control that would be needed. Due to the monospace appearance of these reports, however, \datanull reporting seemed to be out as well.

Fortunately, \ODSDSO's ability to blend the layout control of \datanull reporting with the formatting control of traditional ODS would help save the day. with \ODSDSO, SAS programmers can turn raw data into production quality reports ready for distribution. The \ODSDSO\ is a powerful Base SAS feature that is available (in preproduction) as of version 9.1.3 and is able to generate production quality PDF reports directly since it provides SAS programmers simultaneous control over report layout and formatting.

This is the first in a series of papers that will document the evolution from a reporting process that is currently only 10\% SAS to one that approaches 100\% SAS.

The code discussed in this paper was developed with SAS 9.1.3-SP4 running under Windows XP Professional-SP2. The code was developed specifically for the PDF destination, you may experience some nuances when applying these techniques to other destinations.

Keywords: \SUGconfKeywords
\end{abstract}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{DATA NULL + ODS \textless\ \ODSDSO}%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{floatingfigure}[r]{.45\textwidth}
    \fbox{\includegraphics[width=.4\textwidth]{class.pdf}}
    \caption{Simple Report for SASHELP.Class}
    \label{fig:Class}
\end{floatingfigure}

The \ODSDSO\ is an object-oriented approach to building reports--Daniel O'Connor's SUGI 28 paper provides a great background on the \ODSDSO.

All this controll doesn't come without cost to the programmer. The syntax is quite particular with ordering of statements and micro-coding of the structure and format of the report.

If you've worked with HTML documents then you're already familiar with the idea of opening and closing tags. But HTML is a forgiving language; if you forget to close a paragraph, it will do it for you. Same with list items. XML, on the other hand, is very strict with its structure. Open tags must be closed and closed in proper order. If not, you're be stuck with a document that is not `well-formed'.

In this sence, \ODSDSO\ is much closer to XML than to HTML. Methods are called to start and end layouts, tables, table heads, table bodies, table feet, rows, and cells. They must also end in proper order (i.e., in the reverse order in which they were started). For each unit in the report (the methods), a set of arguments can be called. Some arguments are required for certain methods while most are optional. Most arguments are relevant to a handful of methods and are irrelevant for other methods. If you try using an argument that isn't appropriate for a given method, SAS will throw an error (hopefully the helpful variety) and halt the DATA step.

To make the simple report in Figure \ref{fig:Class}, required a relatively convoluted program given---around 80 lines to create nineteen 5x2 cell tables with some leading and trailing text.
\lstinputlisting[caption={Simple Report for SASHELP.Class}, label={lst:Class}]{class.sas}

\subsection{Using Macros to Reduce Repetitious Code}
After reviewing Listing \ref{lst:Class}, you will probably notice that there is a lot of repetitious between \texttt{table\textunderscore start()} and \texttt{table\textunderscore end()}: start a row, start a cell, format the cell, end the cell, start a cell, format the cell, end the cell, end the row, start a row\ldots and the cells and text share some common options. If you micro-code a lengthy \ODSDSO\ report, you'll quicky get pulled into the minutia; Macro-coding (with macros and macro variables, of course) lets you keep your eye on the big picture while removing redundancies and isolating formatting.

Listing \ref{lst:Macros} shows the reporting macros and macro variables that were used. For each element in a list fed to the \texttt{\%cutRows()} macro, the \texttt{\%varRow()} macro is called and generates the row and cell starts and ends along with standardized formatting (\texttt{inhibit}s and \texttt{override}s).
\lstinputlisting[caption={Reporting Macros}, label={lst:Macros}]{report_macros.sas}

Each time \ODSDSO\ code is submitted, SAS reminds us that the ODS ``data step interface is experimental in this release''. When working with preproduction features, it's a good idea to save often (every few minutes isn't too often) and submit more often (after each marginally significant change to confirm everything runs as expected). Windows users can use \href{http://www.mogware.com/FileHamster}{FileHamster} which automatically copies versions of the file at each save point. When you're beginning to work with \ODSDSO, errors tend to creep into code pretty fast. If you forget to save and submit for too long, weeding out errors quickly becomes a Sisyphean task.

As luck would have it, the class report (Listing \ref{lst:Class}) is one of those cases where too much was changed between runs and an error snuck in---did you notice Alfred's information was missing in Figure \ref{fig:Class}? Since the listing was intended to show the repetitious code, I let the error stand to illustrate my point of saving often and submiting frequently.

\subsection{Two Types of Errors}
Occasionally, the error messages SAS generates are quite helpful. For example, this error message told me that \texttt{column\textunderscore span} was not a valid option for the \texttt{override} argument and proceeded to give me a listing of the options that were valid:

\lstset{language=,numbers=none,frame=none}
\begin{lstlisting}
ERROR 22-322: Syntax error, expecting one of the following: ;, ABSTRACT, ACTIVELINKCOLOR, ASIS, BACKGROUND,
BACKGROUNDIMAGE, BODYSCROLLBAR, BODYSIZE, BORDERCOLOR, BORDERCOLORDARK, BORDERCOLORLIGHT, BORDERWIDTH, BOTTOMMARGIN,
BULLET, CELLPADDING, CELLSPACING, CONTENTPOSITION, CONTENTSCROLLBAR, CONTENTSIZE, CONTRASTCOLOR, CSSSTYLE, CSSTEXT,
DESCRIPTION, DROPSHADOW, ENDCOLOR, FILLRULEWIDTH, FLYOVER, FONT, FONT_FACE, FONT_SIZE, FONT_STYLE, FONT_WEIGHT,
FONT_WIDTH, FOREGROUND, FRAME, FRAMEBORDER, FRAMEBORDERWIDTH, FRAMESPACING, GRADIENT_DIRECTION, HREFTARGET, HTMLCLASS,
HTMLCONTENTTYPE, HTMLDOCTYPE, HTMLID, HTMLSTYLE, IMAGE, IMAGESTYLE, INDENT, JUST, LEFTMARGIN, LINESTYLE, LINETHICKNESS,
LINKCOLOR, LISTENTRYANCHOR, LISTENTRYDBLSPACE, MARKERSIZE, MARKERSYMBOL, MINOR, NEUTRALCOLOR, NOBREAKSPACE, OFFSET,
OUTPUTHEIGHT, OUTPUTWIDTH, OVERHANGFACTOR, PAGEBREAKHTML, POSTHTML, POSTIMAGE, POSTTEXT, PREHTML, PREIMAGE, PRETEXT,
PROTECTSPECIALCHARS, RIGHTMARGIN, RULES, STARTCOLOR, TAGATTR, THRESHOLD, TICKDISPLAY, TOPMARGIN, TRANSPARENCY, URL,
VISITEDLINKCOLOR, VJUST, WATERMARK.
\end{lstlisting}

More often, however, the error messages are somewhat less than useful:

\begin{lstlisting}
ERROR:  Read Access Violation In Task [ DATASTEP )
Exception occurred at (60DA1222)
Task Traceback
Address   Frame     (DBGHELP API Version 4.0 rev 5)
60DA1222  03B3F880  sasmrk:mcn_main+0x50222
60DA1046  03B3F8B4  sasmrk:mcn_main+0x50046
60D65210  03B3F950  sasmrk:mcn_main+0x14210
60C1E04B  03B3F998  saspso:mcn_main+0x1D04B
60D62C20  03B3FA90  sasmrk:mcn_main+0x11C20
60C1D2A3  03B3FAE0  saspso:mcn_main+0x1C2A3
60D6127F  03B3FC20  sasmrk:mcn_main+0x1027F
60D556F8  03B3FCBC  sasmrk:mcn_main+0x46F8
60D551A4  03B3FCE4  sasmrk:mcn_main+0x41A4
67A06B0A  03B3FD48  saswob:mcn_main+0x5B0A
60E73E43  03B3FE5C  sasdsi:mcn_main+0x2E43
6725B296  03B3FEC0  sasdsx:mcn_main+0x2A296
05DA9661  03B3FEE4  0003:00009661 <unknown>
01307DB5  03B3FF04  sashost:Main+0xB1125
6723158A  03B3FF44  sasdsx:mcn_main+0x58A
673115BC  03B3FF88  sasds:mcn_main+0x5BC
01262B02  03B3FFA0  sashost:Main+0xBE72
01266C20  03B3FFB4  sashost:Main+0xFF90
7C80B683  03B3FFEC  kernel32:GetModuleFileNameA+0x1B4

ERROR: Generic error in the XOB subsystem.
\end{lstlisting}
\lstset{frame=tb, basicstyle=\scriptsize \ttfamily, upquote=true}

Saving often and submitting frequently can help reverse programming errors.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{The Source Data}%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Submissions from a graduate survey will serve as the actual source data for the actual report, for these examples, however, fictitious data was used. The granularity of the source data is one observation per graduate with each survey item represented by one variable.

The immediate goal in programming this process is to automate the current reporting needs: upper management needs to see results aggregated by school and degree (by `program'). As described above, this was a tedious process of transcriptionresulting in one-page summary sheets for each program. In its current state, this process is definately not scalable, but by programming the full process from raw data to aggregated data to final reports, the process can easily be scaled out when finer aggregations are requested.

\subsection{Preparing the Data}
The source data set needs to be summarized and aggregated by program. This is handled by a custom macro that aggregates one variable at a time to a separate data set. These data sets are then combined into a single data set; this is the data set used for the report-generating data step. There are a variety of item types in the survey, and each item type is aggregated according to different rules. For example, Likert-type response items are aggregated as top responders / valid responders (see Listing \ref{lst:Aggregate}). The resulting data set contains the by variables and two sets (one for the subgroup and one for the population as a whole) of three variables (the count of top responders, the total count of responders, and the percent of top responders).
\lstinputlisting[caption={The \texttt{\%aggregate()} Macro}, label={lst:Aggregate}]{aggregate.sas}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{The Report}%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The structure of the reporting data step will depend heavily upon the the structure of the aggregated data and upon the desired final report structure. Since the desired report structure should closely resemble the structure of the existing report, it seems appropriate to describe the current report now.

\begin{description}
  \item[Pages] One page per subgroup; subgroups will be determined via by variables to increase scalability.
  \item[Subgroup Description] Provide an explanation of the subgroup (what school and degree) along with response count, rate, and graduation time span.
  \item[Table Rows] Heading rows and data rows with one key survey item consuming one row in the report.
  \item[Table Columns] Specify the key survey item along with quantification of the current subgroup and population comparison.
  \item[Notes] Some of the key variables are aggregated differently. These differences are specified at the end of each page along with a date stamp.
\end{description}

\pagebreak
\lstinputlisting[caption={The New Report Code}, firstline=2,label={lst:New_Report}]{report.sas}

\begin{center}
  \begin{minipage}{.48\linewidth}
    \fbox{\includegraphics[width=.9\linewidth]{2005Q1SOE-MS.pdf}}
  \end{minipage}
  \begin{minipage}{.48\linewidth}
    \fbox{\includegraphics[width=.9\linewidth]{report.pdf}}
  \end{minipage}
\end{center}
\label{current_new}
Here you can see the original report (left) and the report up to this point (right). The appearance of the new report is not intended to be a perfect matchto the appearance of the current report, but rather a close approximation. In the next part of this series, I will walk through the dynamic selection of variables for reporting which should hopefully end with the desired report created 100\% in SAS. Stay tuned.

It should again be noted that fictitious data was used in the creation of the pictured reports.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{End Matter}%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Some Resources}
Hamilton, J. (2004). \textit{Using New Features in ODS to Create Master/Detail Reports}. SUGI 29, Montr\'{e}al, Qu\'{e}bec, Canada.
{\small \url{http://www.excursive.com/sas/p229-29.pdf}}

Koopmann, R. Jr. (2007). \textit{\ODSDSO\ 101: Tip Sheet}\\
{\small \url{http://www.sascommunity.org/mwiki/images/b/be/ODS_DATA_Step_Object_Method_101-Tip_Sheet.pdf}}

Ladan, A. I. (2006). \textit{The Absolute Nitty-griddy of ODS Layout: Part I}. SESUG 2006, Atlanta, GA.\\
{\small \url{http://www8.sas.com/scholars/Proceedings/2006/Posters/PO02_06.PDF}}

O'Connor, D. (2003). \textit{Methods for the ODS DATA Step Object}.\\ {\small \url{http://support.sas.com/rnd/base/topics/datastep/dsobject}}

O'Connor, D. (2003). \textit{Next Generation \datanull Report Writing Using ODS OO Features}. SUGI 28, Seattle, WA.
{\small \url{http://www2.sas.com/proceedings/sugi28/022-28.pdf}}

\subsection{Contact Information}
Your comments and questions on this paper and on my \ODSDSO\ tip sheet are valued and encouraged. Feel free to contact the author:\\
\begin{tabular}[t]{rll}
Name & Richard Koopmann Jr. \includegraphics[height=15pt]{../Certification/bp9.jpg} \includegraphics[height=15pt]{../Certification/ap9.jpg}\\
Company & \href{http://www.capella.edu}{Capella University} \\
E-mail & richard [dot] koopmann [at] capella [dot] edu\\
\href{http://www.sascommunity.org}{sasCommunity} & \href{http://www.sascommunity.org/wiki/User:RichardK}{User:RichardK}\\
\end{tabular}

\subsection{Colophon}
This document was typeset in \LaTeX\ using the following document class, packages, and editor:
\begin{description}
  \item[\href{http://www.ctan.org/tex-archive/help/Catalogue/entries/sugconf.html}{sugconf}] A custom document class for SAS User Group Conference proceedings.
  \item[\href{http://www.ctan.org/tex-archive/macros/latex/contrib/listings/}{listings}] Include source code from the source files directly into \LaTeX\ document\dots can Word do that?
  \item[\href{http://tug.ctan.org/dviware/catdvi/enc/textcomp.enc}{textcomp}] \LaTeX\ automatically converts straight quotes (which SAS likes) to curly quotes (which SAS does not like). This package reverses this effect to make included code more portable through copy-paste-run.
  \item[\href{http://www.ctan.org/tex-archive/macros/latex/contrib/graphicx-psmin/}{graphicx}] Allows graphics to be included from a variety of file formats---including PDF!
  \item[\href{http://www.ctan.org/tex-archive/macros/latex/contrib/floatflt/}{floatflt}] Wraps text around graphics to conserve space.
  \item[\href{http://www.ctan.org/tex-archive/macros/latex/contrib/hyperref/}{hyperref}] Fine-tunes PDF (document properties, link properties, etc.).
  \item[{\href{http://www.pspad.com}{PSPad}}] An extremely powerful freeware text editor that supports over 100 different programming and scripting languages along with syntax highlighting (for Windows only).
\end{description}

\SASisRegisteredTrademark

\OtherTrademarks

\end{document}
